PHILOSOPHICAL 
TRANSACTIONS 

OF 

THE ROYAL 
SOCIETY 

rstb.royalsocietypublishing.org 




Research 



a • 

CrossMark 

click for updates 



Cite this article: Overgaard M, Mogensen J. 
2014 Visual perception from the perspective 
of a representational, non-reductionistic, 
level-dependent account of perception and 
conscious awareness. Phil. Trans. R. Soc. B 369: 
20130209. 

http://dx.doi.Org/1 0.1 098/rstb.201 3.0209 

One contribution of 13 to a Theme Issue 
'Understanding perceptual awareness and 
its neural basis'. 



Visual perception from the perspective 
of a representational, non-reductionistic, 
level-dependent account of perception 
and conscious awareness 

Morten Overgaard 1 ' 2 and Jesper Mogensen 3 

1 CNRU, CFIN, MindLab, Aarhus University, Noerrebrogade 44, Building 10G, DK-8000 Aarhus, Denmark 

2 CCN, Department of Psychology and Communication, Aalborg University, Aalborg, Denmark 

3 Unit for Cognitive Neuroscience (UCN), Department of Psychology, University of Copenhagen, Copenhagen, Denmark 

This article proposes a new model to interpret seemingly conflicting evidence 
concerning the correlation of consciousness and neural processes. Based on an 
analysis of research of blindsight and subliminal perception, the reorganiz- 
ation of elementary functions and consciousness framework suggests that 
mental representations consist of functions at several different levels of ana- 
lysis, including truly localized perceptual elementary functions and perceptual 
algorithmic modules, which are interconnections of the elementary functions. 
We suggest that conscious content relates to the 'top level' of analysis in a 'situa- 
tional algorithmic strategy' that reflects the general state of an individual. We 
argue that conscious experience is intrinsically related to representations that 
are available to guide behaviour. From this perspective, we find that blindsight 
and subliminal perception can be explained partly by too coarse-grained meth- 
odology, and partly by top-down enhancing of representations that normally 
would not be relevant to action. 
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1. Introduction 

We still have no generally accepted or comprehensive understanding of the 
mind -brain problem. Human consciousness can be defined as inner subjective 
experiences, such as perceptions, judgements, thoughts, intentions to act, feelings 
and desires. These experiences can be described from a subjective, phenomenal 
first-person perspective. On the other hand, cognitive neuroscience explores the 
neural correlates with respect to brain topology and brain dynamics from an 
objective third-person perspective. This difference in perspectives is on many 
accounts a fundamental part of the reason why consciousness seems incompatible 
with neuroscience. 

This paper will present a new look at how one may conceive of the relation- 
ship between conscious experience and neural processes. It will take as a point 
of departure an analysis of 'blindsight', partly because the phenomenon has 
had a big impact on already existing mind -brain models, and partly because 
research on the topic holds inconsistencies, the resolution of which may be 
crucial to such a theory. 

Many existing models have explicitly attempted a 'creationist 7 strategy, 
trying to explain how consciousness arises from or is created by brain processes. 
As often pointed out (e.g. [1]), it is very difficult, if not impossible, to show that 
a particular relationship between neural and mental states indicates that the one 
level of description explains the other, let alone causes it. To do so, one needs a 
further claim saying more than just two observations go together. 

The reason for our pessimism for the 'creationist 7 strategy is that we believe 
that such a 'further claim' can never be built on the same scientific grounds as 
the correlation. This is a consequence of the fact that the use of one level of 
description, referring to conscious experience, is a necessary methodological 
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requirement to find any correlation [2]. Deducing the com- 
ponents of this level of description from components of 
another (e.g. neural phenomena) that are already mapped 
to the first level is obviously circular. Excluding either the 
correlations or descriptions of conscious states would be logic- 
ally impossible as the neural phenomena (e.g. 'brain areas') are 
only 'components' because they are identified by virtue of 
their correlations with particular experiences. 

These convictions are, however, not the end but rather the 
starting point of our analysis — leading to a framework to 
investigate and possibly understand consciousness in relation 
to neural processes. 



2. Blindsight 

The concept 'blindsight' refers to a 'visual capacity in the 
absence of acknowledged awareness' in relation to a lesion to 
the primary visual cortex [3]. As blindsight is typically con- 
ceived, it is, from the perspective of subjective experience, 
identical to cortical blindness or hemianopia. However, from 
a cognitive perspective it is quite different, as certain visual 
functions are preserved. It seems less important which exact 
functions are preserved, it is rather the very segregation of 
functional and phenomenal aspects of mental states that is 
central to the idea. 

In possibly the first experimental study on blindsight 
(see, however [4,5]), Weiskrantz et al. [6] reported experiments 
with the now famous patient DB. DB was asked to shift his eyes 
from a fixation point to the position where he would guess a 
light was flashed. The experiment showed a weak correspon- 
dence between target position and eye movement. In a 
second experiment, DB now had to reach for the target with 
a finger instead of relying on eye movements alone. With this 
different method, results showed a very clear correspondence 
between target and finger position, especially for larger 
stimuli. Further experiments studied DB's ability to discrimi- 
nate between two possible stimuli (X versus O, horizontal 
versus vertical lines, etc.) and found that he was able to do so 
well above chance level with increased performance as a 
function of stimulus size. 

Many experiments have demonstrated how blindsight 
patients seem able to do complicated tasks in the reported 
absence of conscious experience, but also the limits of the 
visual abilities. Kentridge et al. [7] found that patient GY's 
attention could be directed by cues in the residual part of the 
visual field as well as in the blind field. Cues in the blind 
field could even direct his attention to locations in the healthy 
field, from which the authors conclude that spatial information 
selection and conscious experience rely on different processes, 
and that blindsight patients may have fully or partially intact 
visual attention without visual experience. It is repeatedly 
found that blindsight differs from normal vision by needing 
cueing and prompting for the patient to react to the visual 
information [8]. 

This finding is not specific for blindsight, but is generally 
the case for experiments on unconscious vision in healthy sub- 
jects. Many psychological experiments have attempted to 
prove the existence of unconscious perceptual processes by 
demonstrating that stimuli are perceived when subjects are 
not consciously aware of the stimuli. The basic strategy in 
these studies is to establish conditions under which conscious 
experience does not occur and then to demonstrate that stimuli 



can nevertheless be perceived under these conditions. The suc- 
cess of these studies depends crucially on two factors: first, the 
acceptability of the method used to establish the absence of 
conscious perception, and second, the method of assessing 
that the not consciously experienced stimulus is indeed uncon- 
sciously perceived. Today, research on unconscious vision in 
healthy subjects and blindsight alike is conducted using a com- 
bination of performance measures and introspective reports. 
Unconscious perception is defined as a successful performance 
(e.g. discrimination) in the absence of positive introspective 
reports. Studies based on both types of measures have not 
led to completely convincing results, because it is always poss- 
ible to question whether the measure of conscious perception 
was successful in guaranteeing a complete absence of all rel- 
evant conscious experiences, as absence of evidence is no 
evidence of absence [9]. Nevertheless, the majority of cognitive 
scientists today seem to share the conviction that perception 
comes in two varieties: conscious and unconscious. 



3. Unconsciousness or vague experiences? 

One historically difficult issue for blindsight has been the fact 
that all or most blindsight patients have reported some vague 
experiences associated with the visual stimuli about which 
they are asked to report [10,11]. Many blindsight researchers 
handle such reports by subdividing blindsight into 'type V 
and 'type 2', where 'type V are patients without any con- 
scious experiences related to visual stimuli, whereas 'type 2' 
are patients with conscious experiences related to visual 
stimuli. Such experiences are, however, not conceived as 
being of a visual nature [12]. 

Other experiments seem, however, to support a different 
perspective. Overgaard et al. [13] performed a study on a 
31 -year-old hemianopic patient with left-sided injury to pri- 
mary visual cortex (GR). In the first experiment, letters were 
briefly flashed at different locations on a computer screen 
and GR's only task was to respond to every stimulus, revealing 
that she was blind to everything presented in the upper right 
quadrant. In the second experiment, GR was presented with 
different visual figures and asked (1) which figures were 
shown, and (2) if she actually saw the figure — yes or no. GR 
reported only very rarely that she saw stimuli in the upper 
right quadrant, yet she was able to report correctly about 
these stimuli more often than chance. In the healthy part, 
her reports were significant predictors of correctness, based 
on which the authors concluded that she had blindsight. 
The third experiment was identical to the second, except she 
now should respond with the perceptual awareness scale 
(PAS) with four labelled points, rather than in a binary fashion. 
As a consequence, her blindsight seemingly 'disappeared' in 
the sense that, even though she reported much more vague 
experiences in the upper right compared with the upper left 
quadrant, the relationship between correctness and reported 
experience was identical to what was found within the rest of 
the visual field. All correctness above chance seemed related 
to vague yet conscious vision when using PAS. So, the first 
experiment indicates that GR is a 'cortically blind hemianopic', 
the second indicates that she has blindsight, perhaps type 1, 
and the third that she has blindsight type 2 if blindsight at 
all. One might be tempted to conclude that blindsight is 
indeed closely related to the methods used to study conscious 
perception. This method-dependency is far from being a 



unique feature of Hindsight. Within numerous cognitive 
domains (irrespectively of their relationship to consciousness), 
studies in animal models as well as humans have demonstra- 
ted that radically different conclusions of what is preserved 
or lost after injury to the brain depend heavily on the testing 
methods and experimental procedures applied (e.g. [14,15]). 
We will return to this and other similarities between 
findings within blindsight and problem solving after brain 
injury, respectively, in §5. 

The findings obtained with GR are closely related to 
experiments by Stoerig & Barth [16] investigating GY, one 
of the most thoroughly studied blindsight patients. GY was 
asked to match a visual stimulus presented to the blind 
field with one of different image transformation of the 
same stimulus in the healthy field. With moving stimuli, 
GY accepted the match as long as they were sufficiently 
blurred. The results match with GY's verbal descriptions of 
his 'feelings' as 'similar 7 to that of a normally sighted man 
who, with his eyes shut against sunlight, can perceive the 
direction of motion of a hand waved in front of him. 

The conclusion that the method for reporting has import- 
ant consequences for findings of unconscious perception has 
been reached many times with healthy subjects. The PAS was 
first developed by Ramsoy & Overgaard [17], who asked sub- 
jects to create their own scale for subjective reporting with the 
instruction that they should be able to subjectively experience 
the difference between the scale points they invented. Most 
subjects conformed to a four-point scale categorized as 'not 
seen', 'weak glimpse' (meaning 'something was there but I 
had no idea what it was'), 'almost clear image' and 'clear 
image'. Ramsoy and Overgaard showed that in an 
experimental design where one should expect to find sublim- 
inal perception, subjects were completely at base chance 
when reporting 'not seen'. In a later study, dichotomous 
reports were compared directly with PAS [18]. Using the 
dichotomous report, subjects showed subliminal perception, 
whereas none was present at PAS = 'not seen'. In further 
studies, PAS was found to be more sensitive to subjective 
experience than other four-point scales [19]. Generally, PAS 
fitted better with objective measures such as stimulus 
duration and correctness than did the dichotomous report. 



4. Mental states, representation and gradedness 

Much conceptual work has attempted to make clear dis- 
tinctions between different notions of mental states. Ned 
Block [20] has famously distinguished between phenomenal 
consciousness ('P-consciousness') and access consciousness 
('A-consciousness'). P-consciousness denotes subjective experi- 
ence, such as the taste of coffee, seeing the colour red or thinking 
about a sunny day. A-consciousness denotes states on the basis 
of which a subject is able to reason, and to have rational control 
of action and speech. Hypothetically, the two meanings of the 
term give rise to two meanings of unconsciousness as well, 
namely 'P-unconscious' (yet potentially still A-conscious) and 
'A-unconscious' (yet still P-conscious). The latter of the two 
ideas is the most controversial, and to many scientists it is con- 
ceptual only. The first idea, A without P, is, however, the 
important case in blindsight. If occipital lobe injured patients 
are able to perform certain judgements based on visual stimuli 
without reports about having experienced those stimuli, this 
may satisfy this idea. 



However, experiments such as the ones described above 
challenge the idea that mental states come as 'on'- or 'off-ver- 
sions of states with otherwise identical contents. Instead, they 
suggest a graded account with many stages or 'levels'. This 
is, however, not at all an unusual situation in neuropsychology. 
When one scrutinizes clinical results, patients who lose particu- 
lar functions or abilities as a consequence of brain injury rarely 
lose all aspects of those functions, and rarely lose accompany- 
ing experiences as well. Patients with working memory 
dysfunctions after neural injury do not lose working memory 
altogether [21]. Patients with prosopagnosia after, typically, 
injury to the fusiform gyrus, do not experience 'nothing' 
where there should have been a face, but rather 'something 
unfamiliar' [22]. In other words, it is, in a way, already well- 
established that many cognitive functions come in degrees. 
Still, however, the 'on- off interpretation of mental states is 
often reflected in theories about the neural correlate of con- 
sciousness that look for particular areas or processes in the 
brain that are associated with the 'on' state (e.g. [23]). This 
type of approach considers particular areas or brain processes 
as 'add-ons' to the 'off versions of the identical mental states, 
and therefore seems related to the notion of 'consciousness aris- 
ing' from those particular brain processes. 

Recent theoretical developments, such as the 'partial 
awareness hypothesis' offer alternative ideas [24]. Here, it is 
argued that 'access', as in Block's A-consciousness, should be 
understood as a hierarchy of representational levels where 
each level may be accessed individually. Our proposal shares 
some of the basic assumptions with the partial awareness 
hypothesis, e.g. that information in the brain is processed in a 
hierarchical manner from very simple features of, say, visual 
objects (e.g. flickers of light, contours or movements) to 
complex conglomerates of information (e.g. meaning or recog- 
nition). Here, however, we take this idea a step further and 
argue that the representational account of mental states, con- 
scious experience and action stand in a necessary, yet plastic 
and dynamic, mutually fixed relationship. 

All experiments reviewed above suggest that patients as 
well as healthy individuals perform better in various tasks 
when reporting experiences of relevant objects of the task 
than when they do not. The experiments using PAS even 
suggest almost perfect correlations between performance and 
clarity of experience where difficulties could be described as 
a Tag' between two sigmoid curves rather than as fundamen- 
tally different categories [25,26]. On the basis of evidence, we 
propose that there is an intrinsic relationship of some kind 
between a functional account of mental representation and 
conscious experience, so that the more elaborate a mental repre- 
sentation is, the more clearly the particular object is perceived. 
In conceptual terms, the proposal goes against the idea that a 
conceptual division between 'A-' and 'P-consciousness' reflects 
an empirical division. 

By 'intrinsic relationship' we wish to express that we 
believe this correlational relationship reflects some more fun- 
damental, or ontological, relationship. However, for reasons 
stated above, we do not believe that the relationship between 
functional and phenomenal aspects of mind, as well as the 
relationship between mind and brain, can be expressed as 
identity or in terms of reduction. Our choice of words — that 
the relation is 'intrinsic' — relates to David Chalmers' proposal 
that some (yet undiscovered) natural law connects conscious 
experience with information (per se) [27]. In our version, there 
is, obviously, no evidence for a connection between 



experience and 'information', in this very broad sense, but 
rather for a relationship between experience and at least 
one very specific kind of information: representations that 
are available for action, e.g. a report. 

Blindsight fits well into this conception. Whereas some 
experiments that look for relationships between performance 
and graded experience find some level of experience under- 
lying imperfect yet above chance performance [13,16,28], 
other experiments that do not look for gradedness find good 
performance when patients tend to report no experiences 
(e.g. [29]). However, almost all blindsight experiments, as 
well as experiments on unconscious perception in healthy sub- 
jects, make use of prompting, forced choice guessing and no 
information irrelevant for the experiment competing for 
attention. As mentioned, some experiments have shown that 
very weak experiences of 'something being present' correlate 
with a not perfect but above chance performance. The same 
blindsight patients seem unable to use such weak experiences 
in everyday life, or even in more standard neuropsychological 
testing, where they appear completely blind. Together, these 
findings indicate that strong, top-down functions should be 
considered as part of a neuroscientific model of consciousness 
where information that typically is of little or no use as a guide 
for action can be enhanced and selected as a basis for report 
under such prompting conditions. Such top-down 'enhance- 
ment' related to specific tasks or instructions have been 
shown in cognitive training experiments with visual stimuli 
(e.g. [30]). 

Arguably, our reasoning here indicates that the relationship 
between experience and action is so that the information that is 
most available and relevant to action is what is experienced 
most clearly. From this perspective, top-down functions make 
it possible to enhance the analysis or at least availability of 
specific over other information. In this way, the top-down selec- 
tion of particular representations allows for, say, visual search 
in complex scenes or even searching in one's own memories 
of past events. However, in most normal cases, information 
that is represented with most detail and richness allows more 
flexible behaviour without any need for forced choice options. 
We suggest that this explains the findings that have led to 
'global workspace' models of consciousness (e.g. [31]). 



5. Representations as strategies 

Whereas we believe that our proposal can account for all avail- 
able experimental data and, at the same time, of at least our 
own introspective accounts of what kind of information is 
experienced, an important aspect of the review in §§2 and 3 
has so far been overlooked. An intrinsic relationship between 
representation and experience, modulated by relevance to 
action, does not in and of itself explain why experiments in 
blindsight and animal models reveal that results about what 
is preserved after injury depend on the kind of task or testing 
method that is applied. 

Experiments on blindsight have revealed more than 
whether it represents conscious or unconscious vision (or, as it 
seems, something more graded in between). Some experiments 
also give indications of whether the contents of weak experien- 
ces in blindsight are different from contents in normal vision. 
Azzopardi & Cowey [32] performed a signal detection analysis 
on GY's yes /no detection judgements and forced choice detec- 
tion tasks, and found that his sensitivity was significantly higher 



during the forced choice task. This is different from the perform- 
ance of healthy subjects when having 'near-threshold vision', 
indicating that visual stimuli in blindsight are processed 'in an 
unusual way' [32; p. 14 190]. 

In another study, Morland et al. [33] demonstrated that 
the famous blindsight patient GY is able to match some but 
not all aspects of visual stimuli when asked to compare 
stimuli presented to the blind and healthy hemifields. In 
the experiment, GY associated colour and motion correctly, 
but not brightness, based on which the authors argue for 
an anatomical separation of these functions. Morland et al. 
suggest the results indicate that GY has visual awareness of 
those stimulus properties for which he gave correct reports, 
yet, as they somewhat ambiguously put it, he is not visually 
aware owing to lack of or limited processing of brightness. 

In sum, research on blindsight has so far generated results 
similar to certain central aspects of research on 'healthy uncon- 
scious vision', and, at the same time, that are quite different in 
other aspects. 

This situation bears resemblance to results obtained 
in studies addressing the neural and cognitive mechanisms 
of problem solving in brain-injured and post-traumatically 
recovered individuals. Such mechanisms have been most 
thoroughly addressed in studies using animal models (e.g. 
[14,34-36]). In some instances, post-traumatic rehabilitative 
training of a cognitive task can lead to a solution equally profi- 
cient to that seen in intact individuals [37]. Apparently, the 
experimental animals lacking, e.g. the hippocampus or the pre- 
frontal cortex, are nevertheless able to achieve a task solution 
similar to that seen in intact individuals. 

In these cases, the proficiency of recovered, injured subjects 
and normal individuals, respectively, is similar. However, a 
conclusion regarding identical mechanisms of task solution 
does not withstand further scrutiny. Subjecting the ani- 
mals to organic and behavioural/ cognitive 'challenges' (see 
[14,36]) allows a more thorough analysis of the neural and cog- 
nitive mechanisms mediating the task solution. Even when 
normal and recovered brain-injured individuals have achieved 
identical levels of solution proficiency, such an analysis reveals 
that the groups differ regarding both the neural substrate and 
cognitive mechanisms behind the successful task solution 
(e.g. [34-38]). 

6. The reorganization of elementary 
functions framework 

Studies of the consequences of brain injury and post-traumatic 
cognitive recovery provide apparently contradictory results 
and theoretical dilemmas not unlike those found in blindsight. 

At the most general level, one finds the contradiction between 
the concepts of 'localization' of various 'functions' (e.g. [39-41]) 
and functional 'recovery' as seen in animal models (e.g. [14,15, 
34-38]) and clinical studies (e.g. [42-46]). These two phenom- 
ena have a number of parallels to two of the prominent models 
of functional organization within the brain: the 'modular' the- 
ories proposed by, for instance, Fodor [47], Pinker [48] and 
Pylyshyn [49] (see review regarding 'massive modularity' in 
[50]), and the models emphasizing distributed, connection- 
ist networks (e.g. [51-53]). Modularity emphasizes a strict 
functional localization, in which loss of the neural structure 
mediating that cognitive module leads to a cognitive impair- 
ment. On the basis of such a framework it is, however, hard 



to realize how a function may demonstrate a post-traumatic 
functional recovery (although some such models are less categ- 
orical and remain open to at least a certain degree of sparing/ 
recovery, e.g. [49]). On the other hand, the connectionist frame- 
work makes it relatively easier to conceptualize the dynamic 
network reorganizations seen in post-traumatic cognitive rehab- 
ilitation — but can rarely account for the degree and specificity of 
initial trauma-associated impairments (e.g. [34,35]). 

Perhaps less theoretically challenging, but important, are 
some of the issues mentioned in §3: the fact that the presence 
and nature of brain injury-associated symptoms appear to be 
dependent on details of the method of testing (in such a way 
that apparently cognitively similar variants of a given test may 
nevertheless provide completely contradictory results [14,15]); 
and the fact that even for identical injury to the same neural struc- 
ture, the post-traumatic neural and cognitive reorganizations will 
depend on the task facing the individual [34,35]. 

Attempting to establish a comprehensive framework of 
the neural and cognitive mechanisms of post-traumatic 
functional recovery within problem solving, the reorganization 
of elementary functions (REF) framework was constructed 
[34-36,54,55]. The REF framework essentially describes a 
connectionist network in which, however, the 'unit 7 is not a 
neutral and functionally 'indifferent neuron 7 , but advanced 
information processing modules called elementary functions. 
REF is thus able to account for both the localization and 
post-traumatic recovery of functions. 

7. The reorganization of elementary functions 
and consciousness framework 

As emphasized above, we see a number of striking simi- 
larities between some of the findings related to blindsight 
and the results achieved within studies of post-traumatic cog- 
nitive recovery of problem solving after focal brain injury. 
These similarities have prompted us to suggest a variation 
of the REF framework as a possible way to uncover neural 
correlates of consciousness (Mogensen & Overgaard [56]). 
We believe that this framework, called reorganization of 
elementary functions and consciousness (REF CON), can pro- 
vide a comprehensive frame within which presently available 
data can be interpreted. Additionally, our theory will stimu- 
late future research able to clarify many of the outstanding 
issues regarding both blindsight and the mechanisms of 
conscious perception more generally (see also [57]). 

According to the original version of the REF framework 
[34,36,54,55] task solution, be it in the form of overt behaviour 
or mental representation, is achieved via two underlying levels: 
the lower level of the elementary functions and the level of the 
algorithmic strategies. Elementary functions perform basic 
information processing and are localized within restricted sub- 
divisions of neural structures. By contrast, algorithmic 
strategies consist of numerous interacting elementary functions 
and are distributed in the sense that the neural substrate of an 
algorithmic strategy includes both the neural substrates of the 
individual elementary functions and the connections mediat- 
ing the complicated interaction between these elementary 
functions. A given task solution is achieved via the compu- 
tations of an algorithmic strategy. In a later elaboration of the 
REF framework, the level of algorithmic modules has been 
added [35]. Similar to an algorithmic strategy, an algorithmic 
module consists of a number of elementary functions and a 



pattern of interconnections between these elementary func- 
tions, computationally constituting a significantly higher 
level of information processing than that which is achieved 
by an individual elementary function. Algorithmic modules, 
however, differ from algorithmic strategies by not being able 
in themselves to mediate a task solution. The information pro- 
cessing of an algorithmic module constitutes a computational 
'subroutine', which contributes to a number of algorithmic 
strategies. The major distinction between an algorithmic 
module and an algorithmic strategy is that the information pro- 
cessing of an algorithmic module is, in itself, insufficient to 
mediate a task solution. The distinction between an algorithmic 
module and an elementary function is that the information pro- 
cessing of an algorithmic module is more extensive and 
cognitively more specialized than that which is the case for 
an elementary function. 

A given behavioural pattern or task solution may be 
achieved via different algorithmic strategies. Focal brain 
injury will deprive the individual of a substantial number 
of elementary functions and thereby all algorithmic strategies 
that depend upon those elementary functions. Thus, injury 
will lead to behavioural impairments of tasks previous- 
ly achieved via activation of those algorithmic strategies 
(e.g. [34,35]). Subsequent training will, however, be able to 
establish novel algorithmic strategies (using preserved elemen- 
tary functions). In addition, potentially the novel algorithmic 
strategies will be able to allow a task solution with a simi- 
lar proficiency to what was seen pretraumatically. This 
'recovered' task solution is, however, achieved via neural 
and cognitive mechanisms ('strategies') different from those 
present pretraumatically. 

The REF CON framework is based on the same types of 
elementary functions, algorithmic modules and algorithmic 
strategies as the original REF framework. A crucial element 
within the mechanisms of REF CON is the perceptual elemen- 
tary function (PEF) which differs from other elementary 
functions by receiving a more or less direct sensory input. 
'Ordinary' elementary functions have no functional specificity 
reaching beyond their basic information processing (the associ- 
ation of an elementary function with a particular cognitive 
domain is purely determined by the dynamically changing 
input /output relationships of the elementary function). By 
contrast, a PEF is essentially 'prewired' to be associated with 
a sensory modality and thus with a perceptual analysis. 

The PEFs are likely to be localized at subcortical as well as 
cortical levels and are the initial central processing steps for 
the incoming sensory information. It should be emphasized 
that the sensory information has been processed peripherally 
before reaching the PEFs of the brain — visual information 
has, for instance, undergone an extensive processing within 
the retina. Within the somatosensory system, some of the 
PEFs will be 'modules' within the somatotopic maps, while 
PEFs within the auditory modality include 'modules' 
within the tonotopic maps. 

Any sensory input to the brain will result in the activa- 
tion of, and information processing within, a number of 
PEFs. In most real-life situations, a sensory situation will be 
associated with the simultaneous activation of PEFs within 
several modalities. 

The information processing of algorithmic strategies 
is shaped via backpropagation mechanisms emerging as 
a consequence of the situational feedback encountered 
when a pre-existing algorithmic strategy is activated and 



mediates behaviour. This experience-associated feedback and 
backpropagation-associated reprogramming shape the con- 
nectivity between elementary functions in a manner that 
can both modulate existing algorithmic strategies and create 
novel versions of such. The algorithmic modules emerge as 
a consequence of such modulations and reorganizations: 
when multiple algorithmic strategies share a network of 
elementary functions this network has become an algorithmic 
module. Through these kinds of mechanisms, PEFs as well as 
more 'ordinary' elementary functions are combined into net- 
works which constitute the perceptual algorithmic modules. 
These networks (perceptual algorithmic modules) are not in 
themselves able to either direct behavioural activity or form 
the basis of conscious representations. They are, however, 
crucial elements in the process leading to behavioural control 
as well as conscious awareness. 

The perceptual algorithmic modules are organized in 
a hierarchical manner reaching from relatively low-level 
modules (situated close to the initial sensory input) to algorith- 
mic modules which become incorporated in a special type of 
algorithmic strategy termed the situational algorithmic strat- 
egy (SAS). When incorporated in the SAS, a perceptual 
algorithmic module becomes able to direct behaviour and 
potentially reach conscious awareness. 

Whenever a pattern of activation of PEF occurs, the acti- 
vated elementary functions will automatically lead to a 
degree of activation of a number of perceptual algorithmic 
modules of the lowest level of this hierarchy (as shown in 
figure la). Every PEF is an element within a huge number of 
perceptual elementary modules of the lowest level. In 
addition, all of these modules will receive a degree of acti- 
vation when such a PEF is activated. The degree of 
activation of a given perceptual elementary module of the 
lowest level will depend heavily upon how many of its con- 
stituent PEFs are active at that moment in time. For each of 
the, in this way, activated perceptual elementary modules, 
the activation initiates a testing of the degree of coherence 
within the activation of its constituent elementary functions. 
Each of the activated perceptual elementary modules will 
'interrogate 7 all of its constituent PEFs as to their degree of 
current activation. In the case of a high degree of 'mismatch 7 
between the pattern of interrogated and activated PEFs, 
respectively, the algorithmic module in question will not 
reach a significant level of activation. If, however, there is a 
reasonable level of congruity of the current pattern of PEF-acti- 
vation and the pattern 'expected' by the algorithmic module in 
question, a full activation of that module is likely to occur. The 
degree of activation of perceptual algorithmic modules in a 
given situation is also a result of mutual competition and 
inhibition between related perceptual algorithmic modules at 
this lowest level. It should be remembered that the networks 
constituting the perceptual algorithmic modules at a given 
level are heavily interconnected and partly overlapping. 

Within the hierarchically organized perceptual algorithmic 
modules, the modules at the lowest level are constituent 
elements within the perceptual elementary modules of the 
next level. Activated modules at the lowest level will lead to 
a degree of activation of a number of perceptual elementary 
modules at the next level (as illustrated in figure lb). The pro- 
cess determining which of these activated modules at the 
second lowest level will eventually reach full activation is 
essentially similar to what was described with respect to the 
selection of the perceptual elementary modules of the lowest 



level. The algorithmic modules at the second lowest level 
which are best able to 'account for' the pattern of activation 
of the lowest level algorithmic modules will reach a level 
of full activation, thereby activating a number of percep- 
tual algorithmic modules at the next level which will then 
enter into a mutual competition as occurred at the lower levels. 

The number of levels of perceptual algorithmic modules 
through which the perceptual analysis is conducted is deter- 
mined by multiple factors. The type of sensory information to 
be analysed is a factor. But the level of experience of the 
individual, as well as the potential presence of injury or dys- 
function, are other factors impacting upon the hierarchy of 
such algorithmic modules. But no matter what the condition, 
the ongoing activation of perceptual algorithmic modules of 
constantly higher levels will continue until the activated algo- 
rithmic module does not feed into any higher level of 
perceptual algorithmic modules. This may be when an essen- 
tially 'full analysis' of the sensory input has been achieved. 
Or it may be due to levels of algorithmic modules being un- 
available because of the lack of experience with the sensory 
information in question, or injury /dysfunction within the 
perceptual systems. Regardless of the reasons, the ultimate 
(i.e. highest available) level of perceptual algorithmic mod- 
ules activated in a given perceptual process will eventually 
be reached, and represent under the current circumstances 
the most advanced perceptual analysis available. 

It should be noted that at all the levels of perceptual 
algorithmic modules, the selected module represents the 
'best available fit' rather than a 100% fit to the activation 
pattern of the immediately lower level. Thus, the current 
model includes contributions from both bottom-up and top- 
down processes, top-down processes not being in the least 
determined by the pre-existing connectivity within perceptual 
algorithmic modules of various levels. As pre-existing connec- 
tivity is shaped by experience, the bottom-up /top-down 
interplay predicts that the contents of experience in many 
cases will be different between individuals even under the 
same circumstances. 

The progression through which the sensory input causes 
the selection of perceptual algorithmic modules of the lowest 
level and subsequently higher levels, is a constant interplay 
between 'forward' activation and feedback activation interro- 
gating the original activation pattern at the immediately 
lower level. Such a process agrees well with the results 
of Garrido et al. [58]. In an analysis of long-latency event- 
related potentials (ERPs), Garrido et al. [58] found that such 
responses are best explained by an interaction between the 
activation of forward and backward connections within the 
systems in question. This is a different perspective on, but 
in agreement with, findings that visual experience is related 
to feedback signals to primary sensory areas [59]. 

Only the perceptual algorithmic modules of the highest level 
of the above-described hierarchy are (potentially) available to 
the regulation of behaviour and /or conscious experience. 
With well-known material and intact perceptual systems, an 
individual will thus only experience the relatively 'completely 
analysed' percept. All previous steps of the perceptual analysis 
will remain unavailable to behavioural control and/ or subject- 
ive awareness. If, however, injury or dysfunction prevents 
parts of the perceptual analysis, what is available to behaviour 
and /or consciousness can be of a more 'basic' and potentially 
fragmented nature, simply because what before was a 'primi- 
tive' is now the best possible 'highest level representation'. 
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Figure 1. Schematic and simplified representation of central aspects of the REF CON framework, (a) Sensory activation of PEFs (shaded PEFs are activated) leads to 
activation of perceptual algorithmic modules of the lowest level (PAM-lls). Activated perceptual algorithmic modules interrogate' PEFs and the perceptual algo- 
rithmic module with the best correspondence between activated and interrogated PEFs receives full activation, (b) Fully activated perceptual algorithmic modules of 
the lowest level activate a number of perceptual algorithmic modules of the second level (PAM-sIs) which in turn interrogate activated as well as non-activated 
perceptual algorithmic modules of the lowest level. Again, the perceptual algorithmic module with the best correspondence between the interrogated and activated 
entities at the immediately lower level receives full activation, (c) The SAS, including perceptual algorithmic modules of the highest level (PAM-hs): some are left 
unintegrated into the SAS, while some are integrated at a variety of levels. For further details, see §7. 



A similar situation may occur if the stimulus material is 
unknown or presented under such atypical circumstances that 
the higher levels of analysis are unavailable. 

The analysed sensory input represented by the perceptual 
algorithmic modules of the highest level is in such a form that 
it is not available as either the basis for action or conscious 
awareness. To reach such a level, it will have to become part 
of an algorithmic strategy. In the case of perceptual algorithmic 
modules of the highest level, they will have to be integrated into 
a special type of algorithmic strategy called the SAS (figure lc). 

The SAS is a highly dynamic algorithmic strategy that 
reflects the current status of the individual. The defining feature 
of the SAS is that it combines elements within sensory/ percep- 
tual, motor/ behavioural, motivational /planning and other 
systems. Some elements within this network are perceptual 
algorithmic modules of the highest level representing the exter- 
nal world. Other elements within the SAS reflect aspects of the 
internal world, as well as the current goal situation in terms of 
what objectives the individual is attempting and /or intending 
to achieve. Additionally, the SAS contains patterns reflecting 
the ongoing behavioural activities. The SAS is in a constant 
state of dynamic flux, being modified according to changing 
motives, goals, emotional fluctuations as well as sensory stimu- 
lation, thereby being even more dynamic and constantly 
modified than is the case with other algorithmic strategies. 

If relevant to the current situation of the individual, the 
perceptual algorithmic modules of the highest level will be 
integrated into the SAS. Once integrated in this network, a 
perceptual algorithmic module will be able to influence the 
behaviour of the individual. 

The integration of a perceptual algorithmic module into the 
SAS can take numerous forms. Any level of integration allows 
the perceptual algorithmic module to influence the ongoing 
and planned behaviour of the individual. However, only more 
elaborate integration and extensive connections to various 
other elements within the SAS allow the perceptual algorithmic 
module in question to influence the conscious awareness of the 
person. In other words, whether the information associated with 
a perceptual algorithmic module will be consciously available is 
determined by the degree and pattern of connectivity between, 
on the one hand, the perceptual algorithmic module and, on the 
other hand, other components of the SAS. 



8. Reorganization of elementary functions 
and consciousness and the mechanisms 
of consciousness 

The determining factor regarding whether or not the content of 
a perceptual algorithmic module is conscious is not the invol- 
vement of any particular elementary function but rather a 
pattern of computational interactions between the perceptual 
algorithmic module in question and other parts of the SAS. 
Such a pattern is influenced (among other factors) by the task 
currently facing the individual. Thus, for instance, different 
patterns of interactions between elementary functions will be 
associated with situations calling for a direct conscious percep- 
tion or a situation calling for a metaconscious approach, 
respectively. These connectionistic differences are primary fac- 
tors determining the differences between results obtained 
under such circumstances (e.g. [60]). 



A crucial factor in determining whether and to what 
extent a perceptual algorithmic module is integrated into 
the SAS, and thereby is consciously experienced, is how rel- 
evant the percept represented by that algorithmic module is 
to the broadly defined actions of the individual. Such actions 
may be in the form of overt behaviour, but could also be 
'inner action' in the form of thought processes. 

As mentioned as a basic consideration for the REF CON fra- 
mework, conscious experience relates to the 'most analysed' 
information from a functional perspective, i.e. the information 
that is most relevant for or available to action. Thus, REF CON 
would not predict any particular brain area as being necessary 
and sufficient for conscious experience. Perceptual algorithmic 
modules of the highest level integrated into the SAS are experi- 
enced regardless of which elementary functions are involved. 
For this reason, identifying a particular neural process associ- 
ated with a given conscious experience does not lead to 
the conclusion that the absence of this neural process in other 
subjects or animals means absence of consciousness. 

The incorporation of perceptual algorithmic modules of 
the highest level into the SAS is a situational and short- 
term process which has to be mediated by neural processes 
able to manifest themselves with an extremely short latency. 
The types of synaptic and general structural plasticity 
involved in more long-term reorganizations of algorithmic 
strategies cannot mediate such a situational process. 



9. Training of conscious experience 

In blindsight, the fact that visual information is typically not 
used spontaneously but can only become a factor in the behav- 
ioural control when the subject is prompted or given very 
specific instructions (e.g. [8,11]), appears to indicate that in 
these cases, the relevant perceptual algorithmic modules can 
only become integrated into the SAS when an external instruc- 
tion has modified the SAS to be receptive to such an integration. 

Normally, a perceptual algorithmic module of the highest 
level will become integrated into the SAS, and thereby 
become available to the control of behaviour, without any 
deliberate effort from the individual. That is, provided what 
is represented by the algorithmic module in question is rele- 
vant to the current goals of the individual. As mentioned in 
§2, the situation in blindsight appears to be somewhat differ- 
ent. The information potentially available in blindsight is not 
spontaneously used in behaviour. For blindsight to be demon- 
strated, external instructions are needed in combination with a 
deliberate effort on the part of the subject (e.g. [8]). 

The somewhat rudimentarily analysed visual information 
available in blindsight is, according to the REF CON frame- 
work, a reflection of a process in which the visual analysis has 
progressed through only a relatively low number of levels of 
perceptual algorithmic modules. Therefore, the analysis will 
reach its most advanced level, the highest level, at a comparably 
earlier stage of analysis than that which is normally seen. 
Although these earlier levels can become integrated into the 
SAS (and thereby made available to behavioural control), 
such a process requires an unusual degree of top-down control 
and effort. 

As previously described, the degree to which an individ- 
ual is conscious of a stimulus depends on the degree and 
pattern of connectivity between, on the one hand, the percep- 
tual algorithmic module of the highest level representing that 



stimulus and, on the other hand, the rest of SAS. The more 
elaborate this connectivity is, the higher the level of conscious 
awareness of the stimulus will be. 

The studies in which a less dichotomous and more graded 
evaluation of conscious awareness (for instance, using the PAS; 
e.g. [13]) has been applied do, however, indicate that at least a 
certain degree of conscious awareness can be demonstrated 
whenever a stimulus is able to direct behaviour. On such a 
background, it seems that whenever a perceptual algorithmic 
module is integrated into the SAS, it is not only available to be- 
havioural control but also to conscious awareness, at least at a 
rudimentary level. The degree of conscious awareness will 
increase with more elaborate connectivity and thereby more 
elaborate information processing. 

In contrast to these immediate processes of the actual incor- 
poration of the perceptual algorithmic modules into the SAS, the 
mechanisms achieving the incorporation of perceptual algorith- 
mic modules into the SAS can be modified by training. 
In addition, these modifications are likely to be mediated by 
neuroplastic processes similar to those achieved via the backpro- 
pagation mechanisms reorganizing other algorithmic modules 
(e.g. [35]). Using the metacontrast masking experimental set- 
up, it has been demonstrated that training can increase the 
utilization and subjective awareness of stimuli which were orig- 
inally neither able to influence behaviour nor to be consciously 
perceived (e.g. [61]). Thus, training can influence whether and 
even how perceptual algorithmic modules are integrated into 
the SAS. These training effects are likely to have at least some 
mechanisms in common with the demonstrated effects of train- 
ing in the case of Hindsight. In a forced choice procedure, 
performance can be improved by training in both monkeys sub- 
jected to bilateral ablation of the primary visual cortex (VI) 
[62,63] and patients demonstrating blindsight (e.g. [64-70]). In 
blindsight patients, training may not only improve the behav- 
ioural performance, but also the subjective awareness (e.g. 
[71]). In intact individuals presented with subliminal stimuli, 
the subjective awareness of these stimuli appears to be able to 
be increased by training procedures (e.g. [61,72]). 

The effect of 'training' and experience can also be 
demonstrated in contexts which are more likely to cause modi- 
fications within perceptual algorithmic modules (of various 
levels). If part of the somatosensory input is lacking, thereby 
leaving part of the somatotopic representation Vacant 7 , a 
reorganization of this representation will occur. If, for instance, 
a hand is amputated, the contralateral somatosensory represen- 
tation will undergo a change in which the two representations 
(the face and arm, respectively) neighbouring the now Vacant' 
hand area will 'grow 7 and eventually completely cannibalize 
the area previously representing the now amputated hand 
(e.g. [73-75]). Thus, within the somatotopic representation, 
the area previously receiving input from the now absent hand 
will become activated by input from body-parts previously 
unable to activate this part of the somatotopic map. In terms 
of the present model, PEFs which were originally activated by 
input from the hand will now become activated by input 
from, for instance, the face. Although the actual input activating 
the previous somatosensory hand-representation is now orig- 
inating from another body-part, the individual may for a 
period of time still experience input to the brain region in ques- 
tion as originating from the now absent hand (e.g. [76,77]). Only 
at a later point in time, when the individual has undergone 
additional 'training 7 in the form of daily experiences, will the 
sensory input activating the previous hand area be experienced 



as actually originating from its novel source (rather than from the 
amputated hand). In terms of the present framework, the mech- 
anisms of this process are that while PEFs of the previous hand 
area quickly undergo changes activating them by input from face 
and arm, respectively, the perceptual algorithmic modules 
which process the activation of these PEFs are not reorganized 
as quickly. For a period of time, these perceptual algorithmic 
modules are still organized in ways which 'assume' the input 
to the hand-PEFs still to originate from the hand. Subsequent 
experiences, backpropagation mechanisms and reorganizations 
are necessary for these algorithmic modules to allow the individ- 
ual to consciously experience activation of the previous hand 
area as representing input from the body areas from where the 
stimulation now actually originates. 



10. Perception shaped by learning 

A somewhat different example which nevertheless may reflect 
related processes can be found in the patient SB reported by 
Gregory & Wallace [78] and Gregory [79]. SB grew up virtually 
blind. Aged 52 years, SB received corneal grafts to both eyes 
and gained the ability to see. When postoperatively first experi- 
encing vision, SB appeared to be almost exclusively able to 
visually perceive what had been available to tactile exploration. 
In a striking example, SB when drawing a bus (offered to his 
visual inspection) included a type of wheel which was no 
longer used on buses. SB had, however, as a child (using tactile 
exploration) experienced that type of wheel. When later again 
tasked with drawing a visually presented bus, SB now drew a 
wheel which was identical to the one actually present. Appar- 
ently, the visual information offered to SB was originally 
analysed through perceptual algorithmic modules which had 
primarily been constructed and structured using tactile infor- 
mation (while SB was blind). Only through a prolonged 
period of visual experiences were the perceptual algorithmic 
modules of SB reorganized into a structure representing the 
(visual) world of the present day. 

As previously mentioned, perceptual algorithmic modules of 
the lowest level are under normal circumstances likely to receive 
activation from PEFs associated with multiple sensory modal- 
ities. Thus, within the framework of REF CON, multimodal 
perceptual phenomena such as the McGurk illusion [80] can 
easily be accounted for. The McGurk illusion (the fact that audi- 
tory perception of syllables is modified by simultaneously 
watching a face pronouncing a different syllable) reflects the 
fact that even when consciously perceived as a purely auditory 
process, auditory perception of spoken language is in reality a 
multimodal process whenever relevant visual information is 
available. Some of the neural mechanisms associated with this 
multimodality are presently being analysed (e.g. [81]) and such 
an analysis may provide cues regarding the neural networks 
mediating the associated perceptual algorithmic modules. 

Since the perceptual algorithmic modules involved in 
various types of perceptual analysis are constantly subjected 
to use-associated feedback and potentially backpropagation- 
provoked reorganizations, REF CON predicts that plastic pro- 
cesses are able to allow not only reorganizations within a 
given modality (e.g. [76,77]) but also reorganizations reaching 
across sensory modalities, as seen in various types of cross- 
modal plasticity (e.g. [82]). Mancuso et al. [83] emphasize 
that according to most traditional views of colour perception 
it is surprising that gene therapy is able to eliminate colour 



blindness in adult primates. According to most prevailing 
theories, the mechanisms of colour perception become 
'hard-wired' in early development and should not be able 
to use an adult insertion of a peripheral analysis of colour. 
REF CON, on the other hand, includes a number of dynami- 
cally reorganizable levels (the levels of perceptual algorithmic 
modules) between the sensory input to the brain and the 
mechanisms of the actual perceptual experience. Thus, REF 
CON can more easily account for such an ability to use an 
adult's age acquired ability to receive colour-coded input. 

11. Studies of the neural correlate(s) 
of consciousness 

Most current theories about the neural correlate(s) of con- 
sciousness (NCC) suggest some specific neural region as 
necessary, or at least sufficient, for subjective experience. For 
instance, Logothetis and colleagues have found that inferior 
temporal regions along the ventral projection streams from pri- 
mary visual cortex are active in relation to the dominant 
percept during binocular rivalry [84]. On this background, 
they suggest these regions as an interesting candidate for an 
NCC. Rees et al. [85] review a number of experiments and 
argue that activations cluster around parietal and prefrontal 
sites when subjects experience changes in bistable figures, bin- 
ocular rivalry, and when they become consciously aware of 
changes in complex visual scenes. Lamme [59], however, 
suggests that we can only be conscious if there are feedback 
loops to primary visual cortex based on, primarily, ERP experi- 
ments showing that visual information can reach even 
prefrontal areas without accompanying conscious experiences. 

There are more proposals for NCC candidates than those 
which can be mentioned here. The few examples above aim 
to show that such candidates are rather different, and cover 
all cortical lobes. One difficulty for all such proposals is that 
they can only account for a limited amount of findings, and 
they are forced to explain why other experiments, with some- 
times small methodological variations, give rise to very 
different findings. 

One advantage of the REF CON framework is that it does 
not presuppose that there is any 'consciousness region' in 
the brain. Rather, it would predict that the level of cognitive 
representation, related to particular algorithmic strategy, correl- 
ates with consciousness. Even though the elementary parts of 
the model are expected to be localized in different regions, 
there is not one particular constellation of functions in one par- 
ticular kind of strategy that is expected to be 'more conscious' 
than other possible constellations. For the same reason, REF 
CON has no principal problems explaining data that in the 
light of other theories look like conflicting data. 



The REF CON framework shares more similarities with the 
relatively few computational theories of consciousness. Accord- 
ing to Tononi [86], consciousness is identical to 'integrated 
information', i.e. consciousness is the property of a physical 
system, and its quantity can be specified mathematically. Crick 
& Koch [87] argued that there might be a common neural mech- 
anism behind 'the binding together' of information that is 
otherwise separately processed in the brain and mechanisms 
of consciousness. Therefore, they argued that synchronous 
activity in the 'gamma range' is causally involved in conscious- 
ness. Although those theories share with REF CON the idea that 
consciousness is not directly related to particular regions in the 
brain, they are, obviously, based on different assumptions about 
which properties to associate with consciousness. REF CON's 
association between consciousness and 'level of representation' 
is based on empirical data, as argued here and in §§9 and 10. 



12. Conclusion 

The REF CON framework is a theoretical proposal aimed at 
resolving seemingly contradictory evidence in blindsight and 
subliminal perception research. The model has a general 
scope, and is therefore applicable to any aspect of consciousness 
research without modifying the principal ideas. One advantage 
of REF CON is its ability to combine seemingly conflicting 
evidence in one, hierarchical organized model with several 
levels. Conflicting data on blindsight and subliminal percep- 
tion can be resolved essentially by arguing that conscious 
experience is not in opposition to functional aspects of mind, 
but rather, an intrinsic part. Thus, the model opposes the idea 
that we can have 'access consciousness' about something of 
which we are 'phenomenally unconscious'. Methodological 
developments in both fields have questioned the classical 
dichotomy between 'on' and 'off versions of mental states, 
and show that vague perceptions may underlie what seems to 
be 'fully unconscious perception'. The neuroscientific side of 
REF CON suggests a combination of organizational principles 
that are otherwise often presented as opposite ideas, namely 
the idea of functional localization, and the idea of a distributed, 
connectionist system. The model essentially suggests that no 
particular brain area or process is itself associated with con- 
sciousness, but that information analysed to a given 'highest 
level' and then integrated into the SAS is experienced. The 
model obviously predicts close correlations between infor- 
mation that is analysed to this degree and experience. But 
denies that one level of description can be said to create another. 
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